Learning Continuous Depth Representation via Geometric Spatial Aggregator

Depth map super-resolution (DSR) has been a fundamental task for 3D computer vision. While arbitrary scale DSR is more realistic setting in this scenario, previous approaches predominantly suffer from the issue of inefficient real-numbered upsampling. To explicitly address issue, we propose novel continuous depth representation DSR. The heart our proposed Geometric Spatial Aggregator (GSA), which exploits distance field modulated by arbitrarily upsampled target gridding, through geometric information introduced into feature aggregation and generation. Furthermore, bricking with GSA, present transformer-style backbone named GeoDSR, possesses principled way to construct functional mapping between local coordinates high-resolution output results, empowering model advantage shape transformation ready help diverse zooming demand. Extensive experimental results on standard benchmarks, e.g., NYU v2, have demonstrated that framework achieves significant restoration gain compared prior art. Our codes are available at https://github.com/nana01219/GeoDSR.